Embedding Imputation with Self-Supervised Graph Neural Networks

Embedding learning is essential in various research areas, especially natural language processing (NLP). However, given the nature of unstructured data and word frequency distribution, general pre-trained embeddings, such as word2vec GloVe, are often inferior tasks for specific domains because missing or unreliable embedding. In many domain-specific tasks, pre-existing side information can be converted to a graph depict pair-wise relationship between words. Previous methods use kernel tricks pre-compute fixed propagating across different words imputing representations. These require predefining optimal construction strategy before any model training, resulting an inflexible two-step process. this paper, we leverage recent advances neural networks self-supervision simultaneously learn similarity impute embeddings end-to-end fashion with overall time complexity well controlled. We undertake extensive experiments show that integrated approach performs better than several baseline methods.